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ABSTRACT 

The aim of this investigation was to develop a 
justifiable and efficient procedure for locating the top two percent 
of the eighth grade students in an inner-city junior high school. The 
Wise performance IQ was chosen as the criterion measure. Four 
screening measures were used to select students for the WI3C: an 
achievement test (the California Achievement Test 1970 reading and 
mathematics) , a conventional IQ test (the California Test of Mental 
Maturity short form 1963), a "culture fair" test (Raven's standard 
Progressive Matrices) and teacher nominations. , Students scoring in 
the top two percent of the sample on one or more of these measures 
took the Advanced Progressive Matrices and the WISC. The top nine 
students on the WISC performance scale were designated mentally 
gifted, .since four of the nine gifted students were not non^inated by 
any teacher, the use of teacher nominations is not recommended except 
as a supplement to objective tests. A procedure involving the use of 
the SPM followed by the APM is recommended for selecting students for 
the WISC. This procedure would have identified eight of the nine 
gifted students in this study, and would have required the 
administration of only lU WiSCs. (Author) 
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AUTHOR'S ABSTRACT 



The aim of this investigation was to develop a justifiable and efficient procedure 
for locating the top two percent of the eighth grade students in an inner-city junior high 
school. The WISC performance IQ was chosen as the criterion measure. 

Four screening measures were used to select students for the WISC: an achievement 
test (the California Achievement Test 1970 reading and mathematics), a conventional IQ 
test (the California Test of Mental Maturity short form 1963), a "culture fair" test (Raven's 
Standard Progressive Matrices) and teacher nominations. Students scoring in the top two 
percent of the sample on one or more of these measures took the Advanced P rogressive 
Matrices and the WISC. The top nine students on the WISC performance scale were 
designated mentally gifted. 

Since four of the nine gifted students were not nominated by any teacher, the use 
of teacher nominations is not recommended except as a supplement to objective tests. 

A procedure involving the use ^f the SPM followed by the APM is recommended 
for selecting students for the WISC. TiJs procedure would have identified eigjit of the 
nine gifted students in this study and would have required the administration of only 
14 WISCs. 
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CHAPTER I 



INTRODUCTION 



i 

It is hoped that this investigation of methods of identifying mentally gifted students 
in an inner-city junior high school will assist those who are responsible for this procedure 
in the schools. 

Recent legislation in California, permitting the use of special criteria in the 
identification of mentally gifted disadvantaged students, has greatly facilitated the 
establishment of gifted programs in inner-city schools. This enables more minority students 
to benefit from the use of gifted funds which previously were used almost exclusively 
in advantaged schools. With the increasing demand for minority personnel at high 
professional levels, it is essential that talent be located early and enabled to flourish. 

The special criteria in California 

Prior to the 1969 passage of an amendment to the California State Code, the criteria 
which a secondary school student had to meet to be placed in a gifted program were 
scores above the 98th percentile point on both a group intelligence test and on a 
standardized achievement test. These are the criteria specified by section 3821 of title 
5 of the California State Code. Since very few students in disadvantaged schools were 
likely to meet these criteria, gifted programs rarely existed in such schools. 

The 1969 amendment allows students who are culturally disadvantaged and 
underachieving to be identified as "mentally gifted" under separate criteria specified by 
section 3822 of title 5 of the California State Code. According to section 3822 a student 
is considered mentally gifted if "he may be expected, within a reasonable time and with 
appropriate curricular modifications, to perform in school at a level equivalent to that 
of the mentally gifted minors identified pursuant to section 3821." 

The legislation continued by stating that this judgment shall be based on characteristics 
such as "precocious development", "unusual resourcefulness" or "outstanding 
achievements". The only objective measures which are specified as possible criteria are 
"scores at or above the 98th percentile on nonverbal (performance) scores of an individual 
intelligence test". 

Finally, there is a clause which states that the number of students identified under 
the special criteria shall not exceed two percent of the culturally disadvantaged pupils 
within a school district. 
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Assumptions underlying the design of the study 

One of the first requirements of a selection procedure is that each student is afforded 
an equal opportunity to be considered. This requires some kind of initial screening of 
all students in a grade level, about 400 students in an average sized junior high school. 
It is doubtful that a measure of "resourcefulness" or "precocious development" could 
easily be collected for 400 students. A major assumption underlying this study is, therefore, 
that an objective, cognitive test is the fairest method of initially screening all students. 

Many cognitive tests have been justly criticized as being culturally biased. Some 
nonverbal tests, utilizing abstract items equally novel to all students, have been called 
"culture fair". Whether abilities measured by "culture fair" tests are in fact uninfluenced 
by such cultural variables as child-rearing practices is not establisheu. It is nevertheless 
clear that the acquisition of vocabulary, for example, is subject to cultural influences; 
words you never hear or read you never learn. A second assumption made, therefore, 
is that cultural differences are particularly evident on verbal or language tests. The question 
of cultural bias would not be of concern in this study if all students in an inner-city 
school were assumed to be equally disadvantaged on verbal tests. Such an assumption 
is not made however. 

Identification is presumably for the purpose of placement of the student in a gifted 
program. Some decisions about the program must be made clear since they affect the 
identification process. Firstly, what is the aim of the gifted program? Section 3822 states 
that a student is mentally gifted if he may be expected to "perform in school" at a 
level equivalent to that of the regularly identified gifted student. This seems to imply 
that the aim of gifted programs for disadvantaged students would be to produce 
achievement scores at the 98th percentile level. If the aim is high achievement, perhaps 
the selection should be based on an achievement measure. To dramatize the issue, consider 
a student with a nonverbal IQ of 135 but a very low reading level. Should he be placed 
in the program in preference to a student with a 120 IQ but who is one of the best 
readers in his grade? 

This is a decision which schools must make. However, it is in the tradition of special 
education, whether for the retarded, the emotionally handicapped or the mentally gifted, 
to try to meet the needs of the highly deviant individual no matter what problems come 
along with his deviance from the norm. It is therefore assumed here Jthat those of highest 
ability should be located for placement in a mentally gifted program. (Accelerated programs 
should exist for high achieving students.) However, this assumption has some implications 
for the nature of the gifted program. If it is to be a traditional lock-step program, placement 
in it of severe underachievers might damage the rate of progress of the whole program. 
In this case the decision is not at all easy. If individual help is not available for gifted 
but severiy underachieving students, then placement of such a student in the gifted program 
is a risk. It will be assumed here that individual help, "adaptive treatment" to use the 
distinction made by Cronbach and Gleser (Cronbach 1957), is available even if the whole 
program is not individualized and, therefore, that the aim of identification is to locate 
students of highest ability, regardless of their present achievement. 



ERLC 



2 



But can "ability" be measured or are ability measures nothing but achievement 
measures? For the practical concerns of tliis paper it is assumed tha^ a test which measures 
skills specifically taught in the schools can be called an : measure, whilst one 

which consists of items less subject to educational infliit <l be considered an ability 
measure. It is not assumed that any test measures the limit of a child, his genetic 
endowment or innate potential. 

Lastly, it is assumed that the aim is to identify a full two percent of a grade level, 
the maximum petmissible. 



CHAPTER II 
DESIGN OF THE STUDY 



The sample 



The eighth grade of an inner-city junior high school was selected for s^dy. The 
school is approximately 87 percent Black and 10 percent Chicano. Since the eighth ^ade 
consisted of about 400 students, the problem under investigation was the location of the 
eight students of highest abUity. (Eight is two percent of 400.) 



The measures 



ReUance on existing data for screening was rejected for several reasons. First, many 
reconls are incomplete or missing, particularly at the beginning of a school year. Secondly, 
records are often not uniform. Students from different school systems, for example have 
taken different tests at different times. TTiere is no saUsfactory way to evaluate for 
example, a CTMM score in fifth grade against a Lorge-Thomdike score in «xth grade^ 
Finally, maturation rates might vary. Is the student who is average »n the s,xth grade 
ne^es^rily average in the eighth grade or might he have "blo^omed"? For these reasons 
it was decided that only current test scores should be used for screening. 

Four measures were selected for concurrent administration: 

I . The «?t.nH.rd Progressive Matrices (SPM) a nonverbal test which has frequently 
been used as a "culture fair" measure. 

2 The California Test of Mental Maturity (CTMM) 1963 edition. Short Form, 
level 3. The CTMM is a conventional intelligence test widely used in schools. 

3. The r.iifnmm Achievement Tests (CAT) 1970 edition, level 4, form A in 
mathematics and reading. 

4 Teacher nominations. Although teacher nominations have been found to be 
inefficient and ineffective in locating mentally gifted students, this measure is still in 
frequent use in the schools. 

It was originally planned that the SPM, CTMM, CAT and teacher nominations would 
all be investigated as means of identifying mentally gifted students. It was quickly realized 
however, that none of these measures either discriminates sufficiently at the upper end 
of the iore distribution or is reUable enough to be used to select just eight students. 



r,.H that anv serious program for mentally gi'^ted students should 
Moreover it was realized tnat any senuus yiyjB"^ u/icr tn each 

criterion for identification. 

decided upon the WISC performance scaie iv 

becomes one of developing a procedure to ''^l^^^^^^^^;^^',^^^^^^^ so is too 

the WISC. (The WISC takes about an hour of a psychologist s time 
time-consuming and expensive to be given to all students.) 

The proposed selection pro '-'-Hure for the WISC 
be the SPM. 

T.e SPM a.o„e, .o»e.er, - "^'^"Jf, ^^"^^^^ 

the SPM. This test is designed to ditierennaie ciMri> u 
superior ability" (Raven 1965). 

F„«..™o«, .he u« Of a .cona before fina. ^jj"^ ^ ^APM CuW 

be small enough to fit into one roum, the library by a counselor, 

„aae unifo™ for a« ^^^^^::^iiZ7:^::':,i;L:. coum 
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In order to validate the selection procedure, an effort must be made in this 
investigation to locate all those of high ability. Consequently, students scoring high on 
the CTMM, CAT and teacher nominations will receive the WISC in addition to those 
selected by the SPM-APM sequence. 

Summary 

The WISC performance scale was chosen as the criterion for the identification of 
mentally gifted students in the eighth grade of an inner-city junior high school. 

A selection procedure was proposed for locating those students who are to take the 
WISC. This proposed selection procedure involves the administration of the SPM in the 
classrooms to all students followed by the administration of the APM to students who 
score high on the SPM. An examination of the relationship of these scores to the WISC 
scores will be made to see if this selection procedure can be made effective and efficient 
in selecting^tiidents who will score high on the WISC performance scale. 

In this study, students will not only be selected for the WISC by the proposed selection 
procedure but also by the CTMM, the CAT and by teacher nominations. 



CHAPTER III 



prcx;edures 



Before any tests were administered, teacher nominations were collected by asking 
teachers to name *' off-the-cuff" the eight students they considered to be of highest ability. 
Two methods wer3 used to combine the nominations. First, the number of nominations 
received by each student was counted. Secondly, scores were assigned to nominations 
according to the rank the teacher gave the student in his list of eight nominees. A simple 
linear assignment of score to rank was used. The scores for each student were then added 
to yield a "teacher nomination score". 

In Table 1, the composition and timing of the cognitive group tests is summarized. 

The SPM was administered in the classrooms. Examples such as those reproduced 
in Appendix C were shown to each class before the test booklets were distributed. This 
introduction to the test was aimed at arousing interest and obtaining optimal effort on 
the test. A thirty-minute time limit was imposed. (The manual recommends the SPM be 
used as an untimed test, but this is not practical in a classroom situation.) 

The APM was administered to small groups of students summoned to the school 
library. Set I was used as a practice set and items from it were discussed prior to 
administration of set II. The discussion given to each group was the same and is reproduced 
in Appendix C. Set II was given as a power test. The time taken by each student was 
noted. 

The CTMM short form was administered in the classroom. The only way it could 
be given in one fifty-minute class period was by omitting test 5. Test 5, a seven-minute 
test of mathematical word problems, was given to each class the day following the major 
part of the test. Thus, whilst the administration of the nonlanguage section was standard, 
the language section administration was not: test 7 the memory test, was answered after 
a shorter time than usual and test S was answered at a second sitting. 

After the administration of the SPM and the CTMM to all 1 1 classes in the eighth 
grade and the collection of teacher nominations, economy of time required that the sample 
be reduced. In four classes, there had been no students scoring in the top two percent 
on any measure. These four classes were, therefore, dropped from the sample. The CAT 
tests administered to the remaining seven classes were the two reading tests (vocabulary 
and reading comprehension) and two of the mathematics tests (concepts and problems). 



*This small group method of administering the APM is not in conformity with the method 
of administration recommended eariier. It was used because too many students were to 
receive the APM in this investigation to enable all to be tested at once, and because this 
method caused less disruption of classes. 



T 



TABLE 1 

COMPOSITION AND TIMING OF THE SPM, APM 
CTMM AND THE CAT AS USED IN THIS STUDY 



No. of Working Time 

Test Sub«Test Items in Minutes 

SPM 60 30 

APM Set I (practice set) 12 untimed 

Set II 36 untimed 

CTMM I. Opposites 15 4 

non- 2. Similarities 15 

lang. 3. Analogies IS 

4. Numbencal Values 15 / 

CTMM 5. Number Problems 10 7 
lang. 6. Verbal Compre- 
hension (vocab.) 25 5 
7. Delayed Recall 25 6 

CTMM Total 120 37 

CAT Vocabulary 40 10 

reading Comprehension 45 40 

Reading Total 85 50 

CAT Concepts 35 10 

math. Problems 15 13 

Math Total 50 23 
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To select students for the WISC, the scores on the SPM, CTMM, CAT and cn teacher 
nominations were rank ordered. The top eight students on each of these tests were 
scheduled for a WISC. This produced a group of 16 students. Five students were tested 
for other reasons: three because they had high APM scores and two for high scores on 
CAT reading or CAT mathematics. 

The APM was given to students scoring above 48 on the SPM, to students selected 
for Wise testing and to a random sample of students. The random sample was selected 
by choosing every tenth name on an -alphabetical list of the ei^th grade. 

Three classes we.e retested on the SPM after an interval of five weeks to enable 
calculation of a retest reliability measure. 

After administration of the CTMM and the APM, a questionnaire asking for students' 
opinions about the test was given. (The questionnaire is reproduced in Appendix C.) 

The administration of group tests is summarized in Table 2. 

A rough measurement of SES level was made based on father's occupation as listed 
in guidance records and supplemented in some cases by information volunteered in 
interviews. The levels "high", "medium" and "low" are meant to refer to levels within 
this sample, not relative to the population at large. The judgments made are shown in 
Table 3. Information on SES level was collected only for the 22 students who took the 
WISC. 
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TABLE 2 
GROUP TESTS ADMINISTERED 



Test 

STANDARD PROGRESSIVE 
MATRICES (SPM) 

ADVANCED PROGRESSIVE 
MATRICES (APM) 

CALIFORNIA TEST OF 
MENTAL MATURITY, 
SHORT FORM (CTMM) 

CALIFORNIA ACHIEVE- 
MENT TESTS: 

Reading: vocabulary 
and comprehension 
Mathematics : Con- 
cepts and 
problems 



Date 
October, 1971 



November, 
December, 1971 



January, 1972 
January, 1972 



Sample 

total eighth grade 
(11 classes) 

a 10% random sample 
of Ss and 48 others 



December, 1971 total eighth grade 



seven classes 
seven* classes 



N 

367 
82 
308 



179 
207 



t 

-f 



TT— ;— „ed were ".ow .rack" c.,«e, contain, no student who had seo«d 
iJ^l XTofThe .ca. sampte on any previous measure. 
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TABLE 3 



JUDGMENTS OF RELATIVE SES LEVELS FOR THIS SAMPLE, 
BASED ON FATHERS' OCCUPATION 



High 

physician 
supervisor 

administrative assistant 



public school or 

college administrator 



Medium 



shop specialist 



navy hospital 

student worker 
phone clerk 



lab. technician 
electronic technician 



Low 

laborer 

molder 
machinist 



pressor 

bodyman 

gardener 

unemployed 
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CHAPTER IV 
RESULTS 



The results section is divided into four parts; concerned with: 

1. A general overview of the screening measures 

2. The reliability and validity of the SPM 

3. The APM and the WISC 

4. The question of underachievement 

A few notes on terminology may be helpful. Since the aim is to locate the top 
two percent which is about eight students, the top eight students on any measure are 
referred to as the "citerion group" for that measure. Thus, the "WISC performance criterion 
group" means the top eight students using the WISC perf-ormance test scores as the 
criterion. (In situations where there are tied scores, the criterion group may be more than 
eight.) To say that a measure "selected" a student means that the student was m the 
criterion group for that measure. 

Reference will be made to "tracks". These are ability streams within the school. 
The single track I class is the highest ability class. There are three track II classes, four 
track III classes and two track IV classes. One class is a combined track II and III class. 

The "accelerated Algebra program" is a class of about 35 students who take Algebra 
in the eighth grade rather than, as usual, in the ninth grade. 




RFSULTS ■ PART I: A HENERaL OVHR VIKW OF THE SCREENING MEASURES 

The seven eighth grade classes which took all screening tests consisted of 
appro^LtX 200 sC^ in age at the beginning of the school year (SeW 

from 15^ months to 175 months. Mean age was 160 months (13 years, 4 months) and 
^standard de^iltlon was 5 months. Tl.e sample was 50 percent m^e. Ethn.c composition 
was 87 percent Black, 10 percent Chicano and three percent other. 

The test most difficult to administer was the CTMM which involved constant timing 
and i^Lctns ke SPM was very easy to administer and - very -11 r^^^^^^^ 
students. Percent responses on the questionnaires filled .n by students after taking the 
SPM and CTMM are shown in Table 4. 

Twenty-two percent of students indicated on the P°st-test questionnaire that they 
liked the CTMM and 22 percent indicated that they d.d not like it. For the SPM he 
coxresponding figures were 37 pen:ent (liked it) and eight percent (d.d not hke .t). 

WhUst 1 1 percent thought they would have failed on the CTMM if it were graded, 
only one percent thought so on the SPM. 

nut Hhntinn of obtained scores. Table 5 presents the results from the seven eighth 
^ad e clals which took all screenin -g tests. Both the mathematics -d read.ng ac^^^^^^^^^ 
fcores show a significant positive skew indicating a cluster of ^^^'^^^JJ^^'^ 
of the scale and a "long taU" towards high achievement The ^wo "on^^^^^^^^ 
(the SPM and the CTMM nonlanguage test) show, by contrast, significant negative skew 
irdlcadng that on these scales fewer students received low scores and more scores clustered 
at the upper end of the distribution. 

Table 6 shows the cutoff scores which define the criterion group (top two percent) 
on each lest. Note that for both the total sample (Table 5) -^'.X^ZfZ ofZ 
6) the percentUe rank of the mean CIT^M nonlanguage score is higher ^a" Jhat of^^e 
lan^uagrscore ITie difference is considerable in the case of the cntenon group, the mean 
rsTrr the CTMM nonlanguage test is at the 92nd Pe-ntile po™ 
norms whereas the mean mw score on the language section is at the 69th percentile. 

This observation conforms with the assumption that "cultural differences will be 
especially evident on language tests". 

n... c.»tion of the selection pmblem . It was suggested in Chapter 11 that none of 
the sl^lg measures is reliable enou^ or shows sufficient ^iscn— ^ -res to 
be used to select just eight students. This statement is supported by the following 
examination of the actual results. 

Table 6 shows the range of scores of the top eight students on each of the t^ts^ 
A comparison of this range with the standard error of measurement, s^, for each 
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TABLE 4 

PERCENT RESPONSES TO THE POST-TEST QUESTIONNAIRE 



Question 



Response on a Five Point Scale 



Mean 



Strongly 
Negative 



1 



Strongly 
Positive 



1. How much did you like 
this test? 

after SPM 5 
after CTMM 14 

2. Were the instructions 
clear? 

after SPM 4 
after CTMM 5 



3 
8 



2 
5 



55 
56 



29 
44 



14 
8 



12 
12 



23 
14 



53 
33 



3.5 
3.0 



4.1 
3.6 



How well did you 
understand this 
test? 

after SPM 
after CTMM 



2 
4 



2 
6 



29 
37 



17 
22 



51 
31 



4.1 

3.7 



4. If the test were 

graded, what grade do 
you think you would 
get? 

after SPM 
aft '^r CTMM 



11 



14 

36 



28 
27 



42 
25 



15 
1 



3.9 
3.4 



Did you have enough 
time? 

after SPM 
after CTMM 



1 

11 



14 

36 



28 
27 



42 
25 



15 
1 



3.6 

2.7 



Do you geneTiily 
like taking tests? 

after SPM 
after CTMM 



10 

10 



32 
29 



37 
42 



19 
18 



2 
1 



2.7 
2.7 



mo 



TABLE 5 

RESULTS ON THE CTMM, SPM AND CAT: TOTAL SAMPLE 



Mean 
Raw 
Score 





Test 


N 


X 


CTMM 


lang. 


193 


22.5 




nonlang. 


193 


31.5 




total 


193 


54.1 


SPM 




208 


39.9 


CAT 


math. 


207 


18.1 




reading 


179 


39.8 


** 

P < 


.001 







*p < .05 



Rank of X 
in Norms 

Percentile Stanine SD Skewness 



14 3 8.4 ns. 

21 3 9.7 neg.* 

14 3 16.4 n.s. 

37 4 7.1 neg.** 

19 3 7.4 pos.** 

25 4 13.8 pos.* 



TABLE 6 

CRITERION GROUP SCORES ON THE 
CTMM, SPM AND CAT 











Mean 


Rank of X^^g 












Raw 


in Norms 






Cut-off 




for 


Score 






Test 


Score 


Range 


Test 


^cg 


Percentile 


Stanine 


CTMM lang. 


39 


6 


3.6 


41 


69 


6 


nonlang. 


48 


5 


3.7 


50 


92 


8 


total 


83 


8 


5.2 


87 


76 


6 


SPM 


51 


4 


3.0 


53 


96 


9 


CAT math. 


32 


7 


2.9 


35 


73 


6 


reading 


67 


6 


3.7 


69 


83 


7 



mic 15 



indicates that much of the selection could be accounted for by errors of measurement. 
Many of the students scoring below the cut-off score might have true scores above it, 
and conversely, many scoring in the top eight might have true scores which would rank 
them much lower. 

The problem can be illustrated by reference to the CAT mathematics scores (concepts 
and problems test combined) shown in Figure 1. 

In Figure I, the hi^est CAT mathematics scores are graphed, each score being 
represented by an x. If the obtained score is x, the probability ♦hat the true score is 
outside the confidence band given by x i( 1 .96) s^ is less than .05. For the CAT mathematics 
test, the Sg is 2.9. Since (1.96) x (2.9) is approximately 6, a student with an obtained 
score of 32 can be said to have a true score in the interval 32 ± 6 with 95 percent 
confidence. 

It would be unfair to make important decisions entirely on such uncertain data. This 
problem applies to each of the tests considered (in fact, it should be noted that the problem 
is even more severe for the other tests since the ratio Sg range is even larger). As an 
example of how reliable a test would need to be for such a low selection ratio as two 
percent, a test with a standard deviation of 10 raw score units would need a reliability 
of .99 in order that the 95% confidence interval be as small as four raw score units. 
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♦ 



95% 

confldence 
interval 



CAT 
math. 
40 
39 
38 
37 
36 
35 
34 
33 
32 



31 

30 

29 

28 

27 

26 

25 

24 

23 

22 

21 

20 



XXX 

X 

X 



xxxxx 



xxxx 

xxxxxx 

xxxx 

XXX 

xxxx 
xxxx 

XX 

xxxxxx 

xxxxxx 

xxxxxxxxx 

xxxxxxxxxxxxx 

xxxxxxxxxxx 



cut-off to select 



the top eight scores 



Figure 1 

Dlustration of the Selection Problem using CAT 
Mathematics Scores 
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RESULTS ' PART II: RELIABILITY AND VALIDITY OF THE SPM 

ReUability of SPM , Is the SPM as reliable as other available cognitive screening 
measures? Reliability coefficients are the most appropriate way of comparing the 
reliabilities of several tests.* Test-retest reliability is of primary interest in a selection 
problem, but where this coefficient is unavailable, the Kuder-Richardson formula 20 
(KR20) coefficient will be used. (KR20 is the average of all possible split-half coefficients 
and as such is a measure of internal consistency.) 

Three classes (N = 77) were retested on the SPM after an interval of five weeks. 
The Pearson correlation coefficient between the two tests was .86 (p < .001) a respectable 
test-retest reliability. Means and standard deviations for the two tests in this subsample 
are given in Table 7. A plot of scores from the tWo administrations (Figure 2) suggested 
that the relationship is not homoscedastic. The standard error is greater for lower scores 
than for higher scores, implying that higher scores are the more reliable. 

The overall reliability of .86 and the indication that the reliability of higher scores 
is probably greater, make the SPM quite acceptable, with respect to reliability for use 
as an initial screening measure. 

The other tests were not administered a second time, and the only way to compare 
their reliabilities is to consider reliability estimates from the manuals. These are KR20 
measures. No corrections are available for SES or rural/urban differences between this 
sample and the norming populations. However, the reliability estimates can be corrected 
for differences in the variance of obtained scores. Column 5 of Table 8 presents the 
corrected reliabilities for the several tests. 

In evaluating the corrected reliability samples it should be noted that KR20 estimates 
are generally higher than retest measures of reliability. Consequently, the SPM rctest 
reliability coefficient of .86 compares satisfactorily with the CTMM (total) KR20 estimate 
of .89. The most reliable test is, not surprisingly, the longest test, CAT reading. There 
seems to be no reason to reject the SPM on the grounds of unreliability. 

Validity of the SPM . In Table 9, intercorrelations among screening measures are shown. 
The correlations reported indicate that the SPM shows congruent validity. Correlations 
with other ability measures and with achievement measures are moderate, ranging from 
.42 to .60. It does not appear to "predict" achievement as well as the other ability measures, 
the CTMM language and nonlanguage tests. The CTMM language test, for example, 
correlates .65 and .82 with CAT mathematics and reading, respectively. However, this 
is not surprising when it is remembered that the CTMM language test contains a 
mathematics word problems test directly comparable to CAT mathematics problems and 
a vocabulary test (called "verbal comprehension") directly comparable to CAT reading 
vocabulary, (In this sample the two vocabulary measures, CTMM test 6 and CAT "verbal 
comprehension" correlated .81.) Thus, the CTMM language test "surreptitiously contains 
in itself most of the criterion it is claiming to predict" to use Cattelfs words (Cattell 
1963), 
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*Standard errors, being in raw score unit"., are not directly comparable between tests. 
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1 
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30 

Figure 2 
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50 (J 



Graph of Tcst-Retest 
Scores for SPM 
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TABLE 7 



TEST-RETEST RESULTS FOR THE SPM 
N = 77 



Mean 
Score 

t X SD 

I Test 38.1 7.9 

^ Retest 40.4 9.9 



Correlation = .86 



TABLE 8 

RELIABILITIES OF CTMM AND CAT CORRECTED FOR 





SAMPLE VARIANCE 








Norming Population 








Data 


from the Test 




Reliability 






Manual 


SD 


Corrected 








this 


for Samplt 


Test 


'tt 


SD 


Sample 


Variance 


CTMM lang. 


.90 


11.3 


8.4 


.82 


nonlang. 


.81 


8.4 


9.7 


.84 


total 


.91 


17.8 


16.4 


.89 


CAT math. 


.92 


9.9 


7.4 


.85 


reading 


.95 


16.1 


13.8 


.92 



•r = 1 -iLiLlIll^ (Magnusson p. 75) 
Su^ • 
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TABLE 9 



INTERCORRELATIONS (SPEARMAN'S RHO) AMONG 
THE SCREENING MEASURES 
N=: 200 





Measure 


2 3 


4 


5 


6 


1. 


SPM 


.42 .51 


.60 


.49 


.26 


2. 


CTMM language 


.63 


.65 


.82 


.42 


3. 


CTMM nonlang. 




.65 


.60 


.33 


4. 


CAT math. 






.70 


.44 


5. 


CAT reading 








.38 


6. 


Teacher nomination 
score 











NOTE: p < .001 for all figures 
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TTie SPM. on the contraiy . contains not a word, not a number and yet shows predictive 
validity for achievement tests, particulariy in mathematics. 

A factor analysis of the tests should further elucidate "i*"'*jV»'^,f 
Table 10 factor loading, after varimax rotation are shown for the SPM subtests the CTMM 
™es '^^^^^^^^^^ CAT ^sts. Loading above .40 are shown for four f-*- ^^f^ --"j 
for 65 percent of the variance. The f.rst factor could be f ^f:-;"^"^'/^^^^^^^ 
both the CAT mathematics tests and tests 4 and 5 of the CTMM load on t. Test^ 4 
and 5 of the CTMM aie reported in the CTMM manual to constitute a numerical reasoning 
;rctor o^e o^Tour statisLly derived facto, of the CBiM It is on this — 
reasoning factor that the SPM loads most consistently. It also loads on factore HI and 
rvTch C^her seem to compose a factor which the CTMM manual calls logical 
reasoning, measuied by the fust three tests of the CTMM nonlanguage scale. 

THUS, over the total range of sco.es the SPM appears to have ^^^^"^^^'^^ 
which might be called reasoning and which is related, m this sample, to mathematics 
achievement. 

The conelations between tests and the factor analysis indicate aj^ment bctwmi 
the tests over the total range of scores. However, the main concern ^ ^'f ^^^^ 
to screen for students of high ability is its vaUdity at the upper end of the d^tnbution 
What kind of students score high on this measure? Do students who score very hi^ "^J^^ 
po^ess a not very useful specific skUl? How do the students who score highest on the 
SPM compare with students who score highest on other measures? 

In order to answer these kinds of questions the top eight students on |he SPM are 
compared with the top eight students on the cmM. the ^AJ, ^ea^^V^ "T^^^^ 
the Wise. (TTic fuU report of the WISC results is postponed to Part 4 of the results.) 

Table 11 lists the students, represented by letters, in each ^^^tenon jro^p and T^ble 
12 summarizes .ow many "overlaps" there are between these 8"'':.^^ Z^"^^; 
occurring in the criterion group of each of two tests constitutes one overlap for each 
testTon the basis of these two tables the SPM does not suffer m comparison with the 
othi abUity measure, the CTMM. It has more overiaps than the CTMM. and only one 
TZ ZL selected by the SPM is not on any other list. Two of the students selected 
by the CTMM are not on any other list. 

Six of the eight students in the SPM criterion group are in the CAT criterion group, 
that is they are among the highest achievers in their grade. 

It has already been mentioned that the screening measures are not reliable enough 
to se ect the top eight students with any certainty. Nevertheless, these rough compar^ons 
er^e to ndicate that the SPM does not lose validity at the extreme upper end of me 
d'ribu Jn f^^^^ sample of students. Students who scored very high on the nonverbal, 
"culture fair" SPM usuaUy showed other signs of high abhity. 
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TABLE 10 



ROTATED FACTOR LOADINGS FOR SPM, CTMM 
AND CAT SUBTESTS 
N = 118 



Factors 



SPM 



Variables 

Set A 
Set B 
Set C 
Set D 
Set E 



II 



.53 
.42 
.41 

.69 



HI 

.58 

.47 
.63 



IV 

.46 
.49 



CTMM 1 . Opposites 

nonlang. 2. Similarities 

3. Analogies 

4. Numerical Values 

CTMM 5. Number Problems 

lang. 6. Verbal Comprehension 

(vocab.) 
7. Delayed Recall 

CAT Reading Vocabulary 

Reading Comprehension 
Math. Concepts 
Math. Problems 

PERCENT OF VARIANCE 



.75 



.67 
.82 



.45 
.65 
.69 

40.9 



.45 

.81 
.70 

.74 

.63 
.46 

10.4 



.73 
.61 



7.3 



6.3 
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TABLE 11 

THE TOP EIGHT STUDENTS PLACED IN RANK ORDER 
BY EACH OF FIVE MEASURES 









CAT 






CTMM Raw 


Reading 


Teacher 


Wise Fufl 


Score 


SPM Raw + 


Nominations 


Scale IQ 


Total 


Score 


Math. 


Score 


A 


A 


H 


E 


E 


B 


I 


E " 


F 


A 




L _ 


T* 


0 


L 




V* 


J 


A 


U 




F 






I 


;i 


K* 


;i 


1 : 


R* 




B ' 




1 :i 


X* 


H 


H 


:i 




Q 




Q , 









*Student selected by one measure only 
NOTE: Brackets denote tied scores 
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TABLE 12 



OVERLAPS OF CRITERION GROUPS 



Overiaps With 



Criterion 


Total # 

of 
Overlaps 


CTMM SPM 


CAT 


Teacher 
Nominations 


Wise 


15 


3 4 


6 


2 


CTMM 


15 


3 


4 


4 


SPM 


16 




6 


3 


CAT 


18 






2 


Teacher 
Nominations 


11 









*An "overiap" is a student selected by both of two measures. Thus, in Table 3, three 
students are selected by both the CTMM short form and the WISC; two students are 
selected by both the CAT total and the teacher nominations. 
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RESULTS - PART III: THE APM AND THE WISC 



In this section, the APM results are reported for a random sample* and for those 
selected by high scores on the screening tests. The reliability of the APM and its correlation 
with the SPM are examined. The WISC results are then reported, and some characteristics 
of the gifted students (i.e., those in the top two percent using the WISC performance 
scale as the criterion) are noted. 

The effectiveness of the APM in discriminating between high and low WISC scores 
is tested by discriminant analysis, and cut-off scores are selected for the proposed selection 
procedure. 

APM. The random sample which took the APM, consisted of 48 students ranging 
in age from 156 to 181 months. (Mean age was 160 months or 13 years, 7 months at 
the time of testing.) The mean scores of this random sample on the SPM and the CTMM 
were not significantiy different from those of the total sample. The random sample thus 
appears to have been representative of the total sample. 

The scores obtained by the random sample are reported in Table 13.. Mean score 
on the APM was 12.5 with a standard deviation of 4.7. Reliability, for this sample, was 
.79 as measured by analysis of variance (Table 14). 

If students who took the APM because they had obtained high scores on the SPM 
or some other test were included with the random sample, the reliability for this group 
of students was .83 (Table 15). Since the APM is designed for use in the upper ability 
range, it is this figure of .83 which will be used. 

The standard error of estimate for this group was 2.3. Since the range of scores 
for the criterion group was only 5 (the top eight students scored between 22 and 27), 
the discrimination of scores at the upper end of the scale was little better than that of 
the SPM or the other screening tests, as can be seen by referring back to Table 6. 

Table 1 6 shows correlations of the APM with other measures. As was the case with 
the SPM, the APM showed moderate congruent validity in this sample. 

If the APM is to follow screening by the SPM, it is important to know how the 
two are related. In Figure 3, a scatter diagram shows the relationship for the combined 
random sample and selected students who took the APM. The correlation between the 
SPM and the APM was .69. The linear equation for the regression of the APM (Y) on 
the SPM (X) was: 

Y = 0.46 X - 3.7 



Since the APM (1962 edition) has not been widely used, and the manual is quite 
inadequate, the use of a random sample was desirable to establish some baseline data 
and to obtain an estimate of reliability unaffected by restriction of range. 
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TABLE 13 



if 




SCORES ON THE APM, SPM AND CTMM 
OBTAINED BY THE RANDOM SAMPLE 



Test 

APM Set I 

Set 11 

Hme on 
Set II (mins.) 

SPM 

CTMM Total Raw 
Score 



Mean 
8.3 
12.5 
34.1 

38.4 
54.1 



Range 
2 to 12 
4 to 22 
14 to 71 

12 to 52 
18 to 91 



SD 
2.4 
4.7 

14.2 

8.6 
20.1 



TABLE 14 

ANALYSIS OF VARIANCE FOR APM SCORES: 
THE RANDOM SAMPLE 



Source of 
Variation 


Sum of 
Squares 


df 


MS 


Between People 


35.09 


47 


.75 


Within People 


359.86 


1680 


.21 


Questions 


100.85 


35 


2.88 


Residual 


259.00 


1645 


.15 



F 

18.30 



.75 - .16 
rtt = —17- = .79 
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TABLE 15 



ANALYSIS OF VARIANCE FOR 
ALL APM SCORES 

df MS F 

95 .92 47.72 

3360 .23 

35 7.35 

3325 .15 



TABLE 16 

CORRELATIONS (SPEARMAN'S RHO) OF THE 
APM WITH OTHER MEASURES 

Correlation 



Test N with APM 

SPM 81 .69 

CTMM nonlanguage 66 .51 

CTMM language 58 .52 

CAT mathematics 45 .67 

CAT reading 45 .66 



NOTE: p < .001 for all correlations 
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Source of Sum of 

Variation Squares 

Between People 87.06 

Within People 769.81 

Questions 257.40 

Residual 512.41 



_ -92 ' .15 _ 



APM 

25-27 
















III 


1 


22-24 














11 


III 


II 


19-21 












1 


III 


II 


4444 


16-18 








II 




II 


Mi 


nil 




13-15 










II 


II 


444f 
1 


444+ 




10-12 








II 




II 


1 


1 




7-9 






III 


1 


II 


nil 


11 


1 




4-6 


1 






1 


1 










0-3 






1 ' 
















11-15 


16-21 


21-25 


26-30 


31-35 


3640 


4145 


46-50 


51-55 



SPM 

Figure 3 

Scatter Diagram Showing Correlation 
of SPM and APM 
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The standard error of estimate in predicting the APM score from the SPM was 4.0. 



Wise. Twenty-two students received a WISC; 13 male and nine female. A student 
was selected for the WISC if he had scored in the top two percent of the sample on 
one or more of the screening measures. The results, order by performance scores, are 
presented in Table 17. (Note that students are designated by letters which were assigned 
in the order of their fuU-scale WISC scores. Thus, student A had the highest full-scale 
score and B the second highest. B's performance scale score, however, ranked him 12th.) 

In a disadvantaged population a mean measured IQ of 85 and a standard deviation 
of 15 is considered usual. This would imply that the 98th percentile point would be 
116. The fact that the top eight students tested had full-scale scores at or above 115 
suggested that the testing program probably picked out the students scoring in the 
uppermost two percent of the approximately 400 eighth grade students screened. 

In the 22 WISC scores reported here, the performance scores were generally higher 
than the verbal scores, particularly among the higher total scores. Large discrepancies of 
10 points or more (whiih is approximately twice the standard error of measurement for 
the WISC) occured five times in favor of performance scores (discrepancies of 28, 26, 
24, 14 and 1 3 points) but only three times in favor of verbal scores (discrepancies of 
19, 13 and 10 points). This conformed with the assumption that verbal scores would 
be depressed due to cultural differences. 

The WISC performance criterion group . To select the top ei^t students on the 
performance scale required going down to a score of 114 at which point there was a 
*ie so the group was expanded to include nine students, hereafter called the gifted students. 

This group of nine students included seven of the eight students with the highest 
WISC full scale scores. The one student not included was student B. 

Two students, A and G who will be called Albert and Gail, had WISC performance 
scale IQs above 130 and therefore qualified as gifted under the only objective criterion 
specified by section 3822 of the California State Code. 

Student A. Albert was irrepressible. No matter what cognitive test was riven he scored 
in the top two percent. Recognized as blight, he was nominated by his mathematics, 
science, English and music teachers. He is in track /classes and the accelerated Algebra 
program. Hp likes school and wants to go to college and enter a profession. His 
favorite subject is mathematics. 

Gail presents quite a contrast to Albert. She represents a severiy disadvantaged, 
extremely unccracnieving and yet definitely mentally gifted student. 

Student G. Gail's high nonverbal ability was well established by four separate scores: 
the SPM, APM. CTMM nonlanguage and the WISC performance score. On the SPM, 
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TABLE 17 

RESULTS ON THE WECHSLER INTELLIGENCE 
SCALE FOR CHILDREN (WISC) 





Student 


Sex 


Performance 


Verbal 


run sCi 


< 


A 


M 


136 


no 


125 


- 


G 


F 


131 


103 


117 


I 


C 


F 


127 


113 


121 


} 


D 


F 


121 


118 


121 




E 


M 


118 


119 


120 




H 


M 


118 


109 


lis 




I 


M 


117 


108 


113 




F 


M 


114 


121 


120 




J 


M 


114 


no 


113 




M 


M 


113 


100 


107 




T 


F 


111 


87 


99 




B 


M 


110 


129 


122 




K 


M 


110 


109 


no 




R 


M 


106 


97 


101 




L 


F 


104 


113 


109 




N 


F 


104 


109 


107 




0 


F 


104 


lOS 


105 




S 


M 


104 


96 


100 




p 


M 


100 


109 


105 




Q 


M 


97 


110 


104 




U 


F 


97 


100 


99 




V 


F 


94 


m 


99 



t 
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••"♦^ scored 50, missing th criterion ^roup by just one point. On the APM, she scored 
25, the second highest score in the sample. Her IQ score on the CTMM nonlanguage 
scale was the highest in the sample, 13L and the same as her WISC performance 

Her WISC verbal IQ of 103 was based on only five subtests; the vocabulary subtest 
was invalidated since she declined to attempt any of the questions. Her CTMM 
language score yielded an IQ of 5. These items suggest severly depressed verbal 
achievement. She does not like to read, and this leads her to dislike school much 
of the time, especially classes in which she must read and write. Mathematics is her 
favorite subject, and she would like to be a mathematics teacher. On the CAT 
mathematics ^'problems*' test she solved nine of the 15 word problems which ranked 
her tenth in this sample (the highest score was 13 out of the 15), The CAT 
mathematics "concepts'* test, which involved a kind of mathematical vocabulary test 
and loaded on the verbal factor in the factor analysis, pulled her total score down. 
Her total CAT mathematics score was at the 50th percentile point on national norms 
which was well above the average for this sample. Her CA T reading score was at 
the 25th percentile. point on national norms which was the rank of the average score 
in this sample. 

Gail is in track II classes, not in the accelerated Algebra program and was not 
nominated by any teacher. 

The other students in the WISC performance criterion group seemed to fall into two 
categories. There were the high scoring and recognized students (C, E, F and I) who, 
like Albert, received high scores on most measures and were among the top 16 on the 
teacher nomination scores. Then there were the high scoring and unrecognized students 
(D, H and J> \vho were not nominated by teachers although they also received high scores 
on most measures. These three students were not in track I classes or the accelerated 
Algebra program. 

Table 18 shows how students, who took the WISC, scored on other measures. Scores 
on each test have been classified as 2, 1 or 0 according to whether the score was among 
the top eight scores (2), in the top 16 scores but not in the top eight (1), or below 
the cut-off score for the top 16 (0). The reason for the choice of rank 16 as a cut-off 
is a practical one. In California, the state reimburses a school district S40.00 for the cost 
of identifying a gifted student Since an individual test costs about S20.00, the district 
can afford to test about twice as many students as are finally identified as gifted, i.e., 
an efficiency of about 50 percent is financially acceptable. Consequently, if any one 
test is to be used for screening, the pertinent question is how many gifted students score 
in the top four percent on that measure, which in this case means in the top 16. 



♦^^^ . Number of students found to be gifted 

Efficiency - z : :: x 100 

Total number of students tested 
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TABLE 18 (Continued) 



NOTES: (1) "2" indicates the student was in the top eight students. "I" indicates the 
student was in the top 16 though not in the top eight. "0" indicates the 
student scored below the cut-off score for the top 16 students. 
(2) Scores are listed in order of the WISC performance scores. 

*Top scores on this measure 
NA: Score not available 
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Table 18 supports very strongly the use of the SPM as a screening measure. If one 
screening test had to be used alone, the SPM would have been the best in this group. 
The use of any measure other than the SPM or the CTMM nonlanguage test would have 
overlooked Gail and use of the CTMM nonlanguage test would have overlooked three 
other students. 

Table 18 also illustrates the observations already made that, with the exception of 
Gail, the gifted students were achieving highly. Excepting Gail, not one failed to be among 
the top 16 on the CAT mathematics test and the CAT reading test. Scores on the CTMM 
nonlanguage and language scales were less consistent but generally high. 

Some noncognitive characteristics of the WISC performance criterion group . No 
generalizations about the characteristics of gifted students in inner-city schools can be 
drawn from examining one group of nine students. For example, the fact that seven out 
of nine students mentioned mathematics or science as their favorite subjects (Table 19) 
could be a chance effect, could be due to the particular teachers involved or could be 
a typical result. Only a larger sample could allow positive inferences to be made. However, 
when asked why a subject was mentioned as a favorite, the reasons given concerned the 
content of the subject not the characteristics of the teacher. For example, "I like a 
challenge." "I'm learning something new." "I just love working problems." These attitudes 
could be inspired by good teaching, of course, but they could also reflect a preference 
of gifted students for subjects rich in structure and reasoning. 

Very favorable comments were made by students about the accelerated Algebra 
program. Two of the students not in this program, D and H, commented that their 
mathematics classes were too easy. It is likely that, had more students been in the 
accelerated program, the number choosing mathematics as a favorite subject might have 
been even higher. 

Career choices of this group were varied. Two students mentioned engineering in 
terms of the mechanics of automobiles or rockets. Otherwise, career choices mentioned 
were draftsman, doctor, nurse, teacher, commercial artist, secretary, air hostess, football 
player, musican and writer. 

Three of the group, when asked to recall elementary school years, mentioned problems 
of being teased. They showed an awareness of possible causes with remarks such as, "Tm 
the sort of person whos easily hurt." "I was skipped a grade, so I was younger and 
smaller than the other guys." "I used to get mad real easy. For what? If I lost a game!" 
(laughed) 



*Periiaps the only inference justified is a rejection of the hypothesis that, given a group 
of mentally gifted disadvantaged students, the majority will never prefer mathematics and 
science to other subjects. Here we have a group that does. 
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The proposed selection procedure. It had been hoped that the APM would "spread 
out the scores at the top end of the scale enabling decisions about individuals to be 
made which were less subject to test errors. However, as noted above, the scores on the 
APM were as clustered as on the other screening tests and the standard error of 
measurement was as large in comparison with the criterion group range. 

Did the APM, nevertheless, discriminate between those who scored high on the WISC 
and those who scored low? 

To answer this question, a step-wise discriminant analysis was made between hieh 
and low WISC groups. 

The WISC scores were divided into two groups. The high WISC group consisted of 
the nine students who scored 1 14 or higher on the WISC performance IQ scale (i e the 
WISC performance criterion group). The low WISC group consisted of nine students who 
scored below 114. (Students M, S and R had to be omitted from the analysis due to 
missing data.) 

Table 20 shows the mean scores of the high and low WISC groups on the SPM, 
APM, CTMM language and nonlanguage tests and the four CAT tests. 

Except for the mathematics concepts test, all high-WISC means were higher than 
low-WISC means, but only three reached the .05 significance level. The most significant 
difference in mean scores was on the APM. This lent support to the original supposition 
that the APM, the most difficult test, would show the most discrimination. The other 
two tests which showed significant differences between means were the SPM and 
mathematics problems. Thus, the students who scored highest on the WISC performance 
scale were most different from other students who took the WISC in their Matrices ability 
(SPM and APM) and their achievement level in solving mathematical word problems. 

When the discriminant function, using the APM score, was applied to the groups 
15 out of 18 students were correctly classified. This was a promising result. 

Exactly how should the SPM and APM scores be used to select students to take 

T 1'^' ^'^"^ ^^"^ '"^ S^P*'"^ ''tudents who took 

the WISC. The scores of the gifted students are indicated by stars. It can be readily seen 
that all the gifted students scored 21+ (21 or higher) on the APM or 52+ on the SPM. 

In this sample, the foUowing selection procedure would have been both efficient and 
effective: 

1. Test all eighth grade students on the SPM, 

2. Give APM to all students who scored 49+ on the SPM (N=2l). 
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TABLE 20 

MEAN SCORES OF THE HIGH-WISC AND LOW-WISC 
GROUPS ON OTHER MEASURES 







Mean Score 


Mean Score 








or Hign- 


of Low- 






Test 


WISC Group 


WISC Group 


F 


SPM 




51.0 


45.0 


5.12* 


APM 




22.1 


17.7 


13.56** 


CTMM 


Lang. 


35.9 


34.4 


0.14 




Nonlang. 


47.3 


43.5 


1.66 


CAT 


Concepts 


21.0 


22.1 


0.47 


Math 


Problems 


11.1 


7.8 


6.69* 


CAT 


Vocabulary 


32.7 


29.2 


3.19 


Reading 


Comprehension 


32.3 


29.2 


0.92 


NOTE: 


^m,\,l6 = 8.53; 


^.05,1,16 


= 4.49 





*p < .05 



ERIC 



APM 




SPM 



Gifted student 
**Nongifted student 



Figure 4 



Graph of SPM and APM Scores for 
Students who Took the WISC 
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3. Select for the WISC all students who scored 21+ on the APM or who had scored 
52+ on the SPM (N=14). 

The number of students receiving a WISC would have been 14 of which eight were gifted 
(i.e., in the WISC performance criterion group). This is an efficiency of 56% and an 
effectiveness of 89%. 

The one gifted student overlooked by this process was student I who only scored 
45 on the SPM and, therefore, would not have been selected for APM testing. However, 
if students nominated by two or more teachers had been added to the APM testing sample, 
this v^ould have picked up student I. He scored 21 on the APM and would thus have 
been selected for WISC. 

Based on this retrospective examination of WISC and matrices scores, a generalized 
selection procedure is shown in Figure 5. A discussion of the procedure is postponed 
to Chapter V. In Table 21, the scores made by individual students on the SPM, APM 
and Wise performance scale are reported. 



« # of gifted students identified by process 

Effectiveness of a process - z — ~ — T^ZTZ i ^ '^u 

Total # of gifted students 
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1. Administer SPM to 
all students as a 
30 minute class- 
roori test. 




2. Administer APM as 
a power test to the 
top 6% on the SPM, 
Include also students 
strongly recommended 
by teachers. 




APM 1 50%i 



3. Administer WISC to 

a) top 50% of AFM sample 

b) students who were in 
the top 2% on the SPM. 




WISC 




4, The. top students on the 
WISC performance scale 
are the gifted group. f, /\ 



Figure 5 

Recommended Procedure for Identifying 
Mentally Gifted Disadvantaged Students 
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TABLE 21 



APM, APM AND WISC PERFORMANCE SCORES 



b 









WlaL rertormance 


Student 


SPM 


APM 


Performance IQ 


H 


55 


26 


118* 


J 


53 


23 


114* 


T 


53 


21 


in 


E. 


53 


20 


118 


A 


52 


20 


136* 


F 


52 


19 


114* 


0 


51 


22 


104 


y 


51 


19 


97 




50 


27 


NTa 


G 


50 


25 


131* 


C 


50 


24 


127* 




50 


24 


NT 


X 


50 


17 


NT 




50 


14 


NT 


D 


49 


21 


118* 




49 


21 


NT 


Kf 

N 


AQ 


20 


104 




49 


20 


NT 




AQ 


10 


IN 1 


• 


49 


14 


NT 




AO 


Is 


KTT 

IN 1 


M 


48 


27 


113 


S 


48 


22 


104 


L 


48 


17 


104 


I 


45 


21 


117* 


B 


45 


18 


110 


P 


45 


18 


100 


Q 


44 


14 


97 


V 


43 


15 


94 


R 


42 


16 


106 


K 


28 


1? 


110 



a NT = not tested. 

b Down to 49 on the SPM, all scores are reported. Below 49, only scores of students, 

who received a WISC, are listed. 
*in WISC performance criterion group, i.e., gifted. 
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RESULTS • PART IV: THE QUESTION OF UNDERACHIEVEMENT 



To be eligible for selection under the special criteria allowed by section 3822, title 5 
of the California State Code, a student must not only be "culturally disadvantaged" but 
also "underachieving". Underachievement is generally understood as being indicated by 
a disparity between ability and achievement rank order scores. It is important to remember 
regression effects in such comparisons. If a group of students is selected for high percentile 
rank on one test, say test A, and these scores are compared with those on a second 
test, test B, the percentile scores on test B will tend to be lower than those on test 
A. This is because propitious errors will have accounted for some of the high percentile 
ranks on test A, the selection test. It is unlikely that propitious errors favor each student 
again on test B so, consequently, test B scores will generally be of lower percentile rank. 
As a consequence of this regression to the mean, gifted students (selected for high scores 
on an ability measure, test A) generally appear to be achieving, i.e., their percentile ranks 
on achievement measures (test B) are lower than those on the ability measure used in 
selection. 

This effect can be avoided, however, by use of ability scores from a test other than 
the one on which the selection was based. In the present case students were selected 
as gifted on the basis of WISC performance scores. Four other "ability" measures can 
be used for comparison with achievement scores: the WISC verbal, CTMM nonlanguage 
and language, and the SPM. Table 22 shows the percentile ranks of the gifted students 
in the norming populations on these four ability measures and on the CAT achievement 
tests in reading and mathematics. 

Using the WISC verbal scores as the ability measure, every student received a lower 
percentile rank on CAT mathematics. In fact, except for Albert and Gail, every student's 
percentile rank on the mathematics achievement tests was more than 10 points below 
his WISC verbal percentile rank. This pattern of underachievement was not repeated in 
the reading scores however. Comparing CAT reading percentiles with WISC verbal 
percentiles only two students (Gail and student F) showed an ability-achievement difference 
of more than 10 points, and six students received reading percentile ranks at or above 
their WISC verbal percentiles. 

Using the "culture fair" ability measure, the SPM, all percentile ranks on both 
mathematics and reading achievement measures were lower, indicating underachievement. 
with the exception of student I. 

To be rigorous, some measure of the reliability and significance of the differences 
would be needed, but such rigor is hardly warranted here since absolute faith is not placed 
in the ability measures, and the discrepancies are sufficient to meet the needs of the 
state criteria. Suffice it to say that the gifted students showed evidence of underachievement 
particulariy in the field of mathematics and particularly if the SPM were used as the 
ability measure. 
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TABLE 22 



PERCENTILE SCORES OF THE GIFTED STUDF.NTS ON 
ABILITY AND ACHIHVIiMHNT MEASURKS 







Ability Measures 




Achievement 


Measures 






CTMM 


CTMM 


Wise 


CAT 


Ci«T 


Student 


SPM 


Nonlang. 


Lang. 


Verbal 


Math. 


Reading 


A 


90 


98 


79 


74 


73 


74 


G 


90 


98 


18 


58 


50 


25 


C 


90 


93 


50 


81 


56 


85 


D 


90 


46 


66 


88 


56 


81 


E 


95 


79 


84 


90 


79 


90 


H 


95 


82 


66 


73 


60 


85 


I 


75 


84 


79 


71 


56 


74 


F 


95 


93 


62 


92 


1 


79 


J 


95 


76 


54 


74 


60 


81 


« 

erage 


90 


88 


62 


78 


63 


74 



*This is the percentile rank corresponding to the average score, not the average of the 
percentile ranks. For the CTMM scores the average age of the group, 161 months, was 
used in finding the average percentile rank. 

NOTE: These are percentile scores in the norming population for each test. 
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SUMMARY OF RESULTS 



As suspected, none of the group measures showed sufficient discrimination of scores 
to be used alone to select the top two percent. Errors of measurement could account 
for most of the selection. 

The SPM, the preferred screening measure since it is a "culture fair" test, was well 
received by students, easy to administer in the classroom as a thirty-minute test and showed 
a satisfactory test-retest reliability of .86. The mean score of 202 students was 39.9 with 
a standard deviation of 7.1. An apparent lack of homoscedasticity on the test^etest graph 
suggested that the standard error of measurement, 2.6 raw score units, should be less 
for higher scores. 

Correlations of the SPM with the CAT and CTMM ranged from .42 to .60. These 
correlations, a factor analysis and an examination of scores at the top of each screening 
measure, suggested that the SPM measured, in this sample, an ability relevant to school 
achievement, particularly mathematics achievement. 

The mean score on the APM, administered without a time limit to a random sample 
of 48 students with a mean age of 13 years 7 months wn- ^2.5. The range was 4 to 
27. 

Analysis of variance of all APM scores (those of the random sample plus those of 
students selected to take the APM because of ttl ' scores on other measures) yielded 
a reliability estimate of .83. 

Twenty-two students, selected for high scores on one or more of the screenin^; 
measures, took the WISC. Two students received WISC performance scale scores over 130 
thus qualifying them as mentally gifted under the only objective criterion specified by 
section 3822 of illle 5 of the California State Code. One of these students was strong 
in all areas, scoring in the top two percent on every measure given. The other student, 
here called Gail, would have been entirely overiooked had the CAT, the CTMM total 
scores or teacher nominations been the screening measure. Her verbal scores were severely 
depressed (CTMM language I ) of 85) whilst her nonverbal scores were consistently 
outstanding. (e.g., CTMM nonlanguage IQ of 131; APM score of 25.) 

As there was a tie for eighth place on the WISC performance IQ scale, tJ . *'gifted" 
group (defined as the top two percent using the WISC performance IQ as the criterion) 
consisted of nine students. With the exception of Gail these studen^^ were achieving in 
school at or near the top of their peer group on culture fair and conventional intelligence 
tests (the SPM and CTMM, respectively) and on reading and m'\thematics tests (the CAT). 
School grades were generally good though not exceptional. The grade ht average was 
B or better for every student but Gail. Despite this generally high achieven : ' as measured 
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by both standardized tests and school grades, three of the gifted students in addition 
to Gail, were not nominated as possibly gifted by any teacher. Seven out of the nine 
gifted students mentioned mathematics or science as their favorite subject. 

Using the WISC verbal score as an ability measure (to avoid regression effects since 
selection was based on the WISC performance score) there were strong indications that 
the gifted students were "underachieving'* in mathematics but generally not in reading. 
Use of nonverbal ability measures generally showed underachievement in both mathematics 
and reading however. 

The proposed selection procedure (the SPM followed by the APM) could have selected 
14 students for the WISC eight of which were in the gifted group, indicating that it is 
an efficient and effective selection procedure. Although the APM did not "spread out" 
the high scores any better than did the other screening measures, discriminant analysis 
of high and low WISC scores showed it to be the most discriminating test.: Students who 
scored high on the WISC differed most from students who scored low on the WISC in 
their scores on the APM, the CAT mathematics word problems test and the SPM, in that 
order. 
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CHAPTER V 



DISCUSSION OF RESULTS 



In this chapter, the assumptions made in the design of the inves' .nation arc examined 
■■'» the light of the results obtained. The recommended selection procedure is then discussed 
aiid some comments made on the contrasting problems of special and normal criteria. 

The assumptions in retrospect 

Tlie first assumption was that an objective cognitive test for all students is the fairest 
initial screening measure. This assumption was initially justified on the basis that if the 
criterion is to be performance on tests (as section 3822 of the California State Code 
woulc. seem to indicate) the selection procedure should be based on test performance. 
However, the assumption can also be examined by consideration of the alternatives. One 
alternative to an objective cognitive test for all students is to ask for teacher nominations. 
Often these nominations are not used to make final decisions but as a starting point. 
Thus, the folders of the students nominated are then examined, or the students nominated 
are given tests or a committee meets to discuss them. In the present investigation, four 
of the nine gifted students were not mentioned by any teacher and thus would not have 
even entered into consideration had teacher nominations been used as a starting point.. 

Another tempting alternative to testing everyone is to test just the top, track I class. 
Surely the top eight students are to be found among the ones considered to be the top 
35. Yet, five out of the nine gifted students were not in the top track. They were, however, 
m track II classes. 

Nor did grades distinguish the mentally gifted students. Many other students had 
as high or higher grade point averages. 

These findings conform with studies of (he identification of gifted students in other 
populations. Pegnato (1958) for example, studying a high SES junior high school found 
teacher nominations both inefficient (many students were nominated who were not gifted) 
and ineffective (many students who were gifted were not nominated). 

Bearing in mind these results and also the desiraoility of using a selection procedure 
which is above reproach, it seems reasonable to assert that in attempting to locate students 
who score high on an individual test, the screening of all students by an objective cognitive 
test is the best initial step. 
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A second assumption made was that cultural differences will be particularly evident 
on verbal or language tests. This assumption seemed to be well supported for students 
in the upper ability range. Table 3, for example, showed the criterion group mean scores 
on several mersures. The highest percentile rank of these means in the norms was on 
the nonverbal, "culture fair" SPM (96 percentile) followed by the CTMM nonlanguage 
(92 percentile). Thus, the ability of these inner-city junior high school students relative 
to the general population was least depressed on nonverbal measures. 

The pattern of WISC scores supported the same conclusion for students in the upper 
ability range. Differences of 10 points or more between WISC verbal and performance 
scores occured more often in favor of performance scores than in favor of verbal scores 
and the discrepancies in favor of performance scores were larger. 

The third assumption was that the aim of identification is to locate students of highest 
ability, regardless of their present achievement. As it turned out, most students who scored 
high on the culture fair or conventional measures of ability were achieving highly relative 
to their peer group. For example, six out of the top eight on the SPM were in the top 
eight on the CAT total. But the assumption became pertinent in the case of one student, 
Gail, who was decidely outstanding on all nonverbal measures but to whom reading is 
a chore and vocabulary a mystery. Gail, with a WISC performance score of 131 but a 
CTMM language IQ score of 85 (to pick out two contrasting scores) is defmitely an 
underachieving but mentally gifted student. Any screening process which overlooked her 
or students like her should be subject to severe criticism cn that account. 

The recommended selection procedure 

If the WISC sample had been selected only on the basis of the proposed screening 
procedure, which involves only the matrices tests, it might have been biased and conclusions 
could only have been considered apphcable to other samples selected in the same way. 
The question would still remain: were there others who would have also scored high on 
the WISC but who were overiooked by the matrices screening procedure? 

However, 18 out of the 21 students who received the WISC were "selected" by 
measures other than the matrices tests. That is to say. if the matrices had not been used 
at all and students selected for the WISC by only the CTMM total, the CAT total and 
teacher nominations, 18 of the 22 students would have been selected. Consequently, it 
is reasonable to claim that the WISC sample was representative of the highest scoring 
students on several non-matrices measures. 

If it can now be shown, in retrospect, that applying only the recommended selection 
procedure would have been reasonably effective in selecting the gifted group, this will 
be a strong indication of the validity of the selection procedure. 
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Of the 367 eighth grade students who took the SPM*, 21 (approximately six percent) 
scored 49 or over. These students took the ARM. Selecting from this group on the criterion 
of either an SPM score of 52+ (six students) or an ARM score of 21+ (10 students) yielded 
a combined group of 14 students which contained eight of the nine students in the WISC 
performance criterion group. (The one student overlooked was student I.) 

Thus, the matrices tests, althougli requiring no school-learned skills such as reading 
or arithmetic, were nevertheless as good as conventional measures at picking up outstanding 
students. Moverover, a student whose reading and verbal skills were deficient was not 
overlooked as would have been the case with conventional measures. 

The efficiency and effectiveness** of the proposed selection procedure was 89 percent 
and 56 percent, respecrively, in this study These figures are remarkably high in comparison 
with the study by Pegnato (1958). Table 23 reports the highest efficiency and highest 
effectiveness ratios he found. 

There are so many differences between this study and Pegnato's, however, that 
comparisons are really not in order. The data are quoted here because Pegnato *s study 
IS one of the most thorough studies conducted on the location of gifted students, and 
from it the outlook had seemed rather bleak. No screening measure he studied (the measures 
were teacher judgment, honor roll membership, creativity, student council membership, 
mathematics achievement, group IQ scores, and group achievement tests) which had an 
effectiveness above 70 percent, had an efficiency above 23 percent. The highest 
effectiveness ratio he found was 92 percent, but to get this it was necessary to test about 
five students for every one found to be gifted, a very uneconomical procedure. 

In the present study not enough individual tests were given to allow very strong 
conclusions to be drawn, but the results suggest that a procedure involving the 
administration of a test at two levels can be satisfactorily efficient and effective in selecting 
students for individual testing. 

In Appendix D, an attempted cross-validation of the procedure is described. 

Just using the SPM alone, rather than as part of a two stage selection procedure, 
could have selected 13 students for testing of whom seven were in the gifted group, an 
effectiveness of 77 percent and an efficiency of 54 percent. This raises the question as 
to whether the addition of the APM was necessary or justified. The problem which arises 
in the use of the SPM alone is the rapid clustering of scores as the scale is descended. 
To this lack of differentiation must be added the problem of test errors. The SPM alone 
would not be a poor screening instrument, but the addition of the APM permits an increase 
in efficiency and effectiveness and a great increase in the iustifiability and demonstrable 



*This was all eighth grade students present on the testing day. Mean score wis 37.8 with 
a standard deviation of 8.7. 

**Effecttveness is the percent of gifted stude;ib identified out of the total number of 
gifted students. Efficiency is the percent of gifted students identified out of the number 
selected by the screening procedure. 
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TABLE 23 



EFFICIENCY AND EFFECTIVENESS OF THE RECOMMENDED 
SELECTION PROCEDURE (THE MATRICES) COMPARED WITH 
SELECTED RESULT'S FROM PEGNATO'S STUDY 



Study Screening Procedure Effectiveness Efficiency 

Group Intelligence tests 
Pegnato's cut-off 115 92 19 

cut-off 130 22 56 

Present Matrices 89 56 
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fairness of the testing procedure. This is because use of the APM as recommended means 
there is one uniform test administration and students get two chances to score above 
a criterion score, once under timed conditions (the SPM) and once without any time 
pressure (the APM). 

Was there any screening measure other than the matrices which showed equal promise? 
In this regard it must first be remarked that any one of the objective cognitive tests 
used in this study was more efficient and effective in selecting gifted students than were 
teacher nominations. However, every test but the CTMM nonlanguage scale would have 
overlooked Gail who, with a WISC performance IQ of 131, was undoubtedly gifted as 
defined by the special criteria. (The CAT mathematics word problems test would have 
selected her for testing, but it is a brief 15 item test and is usually combined with the 
mathematics concepts test. On the combined score Gail was not among the top 16 
students.) The use of the CTMM nonlanguage test to select 16 students for testing would 
have overlooked gifted students D, E and H. Consequently, on the basis of this investigation, 
the matrices screening procedure appears to be the most preferrable. 

Contrasting problems of special and normal criteria 

The situation in selecting students under the special criteria is rather different from 
the normal selection problem for gifted programs. 

Nonnally, any student scoring above a criterion score, the 98th percentile point on 
an approved test, can be placed in a gifted program and the state reimburses the school 
district. This means the number to be placed in a gifted program is flexible, depending 
only on the number of students who can be found with scores above the criterion. This 
number varies significantly with the socioeconomic status (SES) of the school population. 
In the high SES school studied by Pegnato, for example, (Pegnato 1958) 6.5 percent 
of the school population had Stanford Binet IQs over 136, i.e., at or above the 99th 
percentile point for the general population. Such a school could support a gifted program 
enrolling a large number of students while a less favored school would scarcely have enough 
students qualifying as gifted to make a special program economically feasible. 

In selecting under the special criteria, however, the number to be identified is more 
or less predetermined, being limited to two percent of the total disadvantaged population 
in the school. There is no specified criterion above which a student must score. 
Consequently, there is not the same incentive to search for all the most gifted students. 
No matter what measure is used for identification a school can usually designate two 
percent as gifted under the special criteria. 



51 



n„t il? tl ^ T identification" easier should not lead to less effort being 

put into the Identification process. Decisions based on unreliable test scores, or on 
subjective judgments that are little more than personal "hunches", might Tdo ig Leat 

Zuno W'"' ''''' ' P^^^'- - which 'he tould t pla^ 

Unsupportable decisions deserve to be challenged and in view of current widespread 
dissatisfaction with testing in general, it is possible that they will be 
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CHAPTER VI 
CONCLUSIONS AND RECOMMENDATIONS 



A procedure is recommended for the identification of mentally gifted disadvantaged 
students. 



The procedure involves the screening of all students on the SPM, a culture fair test 
which is easy to administer in classrooms and well received by students. The SPM is to 
be followed by the APM, a higher level of matrices test. Selected students arc then given 
an mdividual test, the WISC, and the performance scale IQ is used as the criterion for 
identification. 



No serious defects were found in this recommended procedure. The SPM appeared 
to be both reliable and valid as an ability measure. The APM permitted discrimination 
to be made between those likely to score high and those likely to score low on the WISC. 
Differentiation of scores on the WISC performance scale was satisfactory and better than 
on any of the group tests. Test scores on conventional ability and achievement measures 
showed the students identified as gifted to be of generally outstanding ability in their 
peer group. 



The assumptions made in the dpsign of the investigation seemed justified or necessary. 
Any objective cognitive test was preferable to teacher nominations in both efficiency and 
effectiveness. Verbal ability scores of high ability students were depressed relative to their 
nonverbal scores. The assumption that the aim of identification was to locate students 
of the highest abUity regardless of present achievement became necessary in considering 
one student. Since she met the only objective criterion specified by the State Code (a 
perf-ormance scale score above 1 30) she should be considered gifted despite very low present 
achievement.. 



The proposed identification procedure appears to be fair and efficient. It would have 
worked well in this sample. 

Need for further research. The use of the WISC performance scale as the criterion 
IS defensible in that an individual test is the traditional method of identifying gifted students 
(because it is considered more reliable and valid than group measures) and a performance 
IQ is specified as a criterion for disadvantaged students by the California State Code. 
Ultimately, however, this criterion needs to be validated against performance in gifted 
programs of various types. 



Further tests of the matrices procedure recommended for selecting students for WISC 
testing are also needed, especially ones involving the administration of a laiger number 
of individual tests to check that highly able students are not being overiooked. 
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Summary of recommendations 



1 . The Wise performance scale IQ score is recommended as the criterion for the 
identification of mentally gifted disadvantaged students. 

2. Reliance should not be placed on teacher nominations or noncurrent test data 
to screen students. 

3. A cognitive test given to all students is recommended as a fair and objective 
screening procedure. In a disadvantaged population a culture fair or nonverbal test is to 
be preferred. 

4. The following procedure, using two levels of the Progressive Matrices tests is 
recommended to select students to take the WISC: 

a. Administer the SPM as a 30 minute test to all students in the grade. This 
can be done in the classrooms by teachers. 

b. Administer the APM to students who scored in the top six percent of the 
grade on the SPM. The APM should be given at the same time and place for ail students, 
U.. in one uniform administration. (NOTE: Students strongly recommended by teachers 
can also be included in the APM testing.) 

c. Select for the WISC students who scored in the top two percent of the 
grade on the SPM or in the top half of the APM sample. 

(A practical, step by step guide to this procedure is given in Appendix B.) 
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APPENDIX A 



SECTIONS 3821 AND 3822 OF THE CALIFORNIA 
ADMINISTRATIVE CODE, TITLE V 



3821. Evidence to Be Studied, Except for Culturally Disadvantaged, 
Underachieving Pupils. Except for culturally disadvantaged, underachieving pupils described 
in Section 3822, among the items of evidence studied by the committee shall be evidence 
described in either (a), (b), or (c): 

(a) A score at or above the 98th percentile on a full scale individual intelligence 
test approved by the Superintendent of Public Instruction and administered to the pupil 
by a person qualified to administer individual intelligence tests. The norm to be used 
for the score is the norm for children of the same age as the pupil tested. 

(b) ror a pupil in grades seven through twelve a score at or above the 98th percentile 
in each of two tests chosen from a list of tests approved for the purpose by the State 
Board of Education and administered by a qualified person to the pupil while he was 
enrolled in grade 7 or above and within 24 months of the date of identification. The 
two required tests are: 

(1) A standardized full-scale group test of mental ability. 

(2) A standardized test of one of the following: 

(A) Reading achievement 

(B) Arithmetic achievement 

(C) Science achievement 

(D) Social science achievement 

(c) The judgments of teachers, psychologists, and administrators and supervisors who 
are familiar with the demonstrated ability or potential of the minors. In any given district 
not more than five (5) percent of the pupils identified under Section 3821 as mentally 
gifted minors shall be identified on such judgments alone. 

3822. Separate Criteria for Identifying Culturally Disadvantaged, Underachieving, 
Mentally Gifted Minors. A culturally disadvantaged, underachieving, mentally gifted minor 
shall be identified by the committee described in Section 3821 as follows: 

(a) As "culturally disadvantaged" - through the committee s study of all available 
and pertinent evidence of a child's language, cultural, economic or environmental handicaps 
that have in the past and may in the future interfere with his success in school, restrict 
the development of intellectud and creative ability, and prevent full development of his 
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potential. The report of the committee shall specify the disadvantage or disadvantages 
to which the pupil is subject. "Cultural disadvantage" shall be distinguished from cultural 
difference; it implies a lack of cultural background rather than a comparison among 
cultures. 

(b) As "underachieving scholastically" • by comparing the pupil's general intellectual 
capacity with this achievement on the basis of all pertinent evidence related to cultural 
disadvantage. Consideration shall be given to each of the following: 

(1 ) The judgment of the committee, all concurring, that the pupil could achieve 
at the upper two percent level were it not for his cultural disadvantage. 

*- 

(2) Test scores revealing discrepancies between general intellectual ability and 
achievement. 

(3) All pertinent school records. 

(c) As "mentally gifted" • on the basis of the judgment of the committee that 
he may be expected within a reasonable time and with appropriate curricular modifications 
to perform in school at a level equivalent to that of the mentally gifted minors identified 
pursuant to Section 3821 . This judgment shall be based upon one or more of the following: 

(1 ) Precocious development and maturation in the preschool or primary period, 
or outstanding scholastic accomplishment at any point in his school career. 

(2) Unusual resourcefulness in coping with responsibilities, opportunities, 
deprivations, problems, frustrations, obstacles, lack of structure and direction, or overly 
structured settings. 

(3) Outstanding achievements, skills, or creative products. 

(4) Scores at or above the 98th percentile on nonverbal (performance) scores 
of individual intelligence tests approved by the Superintendent of Public Instruction. 

(d) In no case shall the minors identified pursuant to this section exceed two percent 
of the culturally disadvantaged pupils (as defined in Education Code Section 6452) within 
the school district. 
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APPENDIX B 



PRACTICAL GUIDELINES FOR COUNSELORS ADMINISTERING 
THE RECOMMENDED PROCEDURE 



1. : Purchase or otherwise obtain the Standard Progressive Matrices (SPM). Cost is 
S2.00 per test. The number purchased should be equal to the largest class size (plus a 
few extra in case of wear and tear or defacement of course.) 

2. Purchase or otherwise obtain the Advanced Progressive Matrices (APM). Cost 
is $1.50 for set I, the practice set and $3.50 for set II, the test. The number purchased 
should be equal to about seven percent of the number to be screened. (For example, 
if the eighth grade is being screened and there ar* 400 students in the eighth grade, purchase 
28 APM*s since 28 is seven percent of 400.) 

3. Have answer sheets prepared. The number prepared for the SPM should equal 
the number of students in the grade being screened. The number prepared for the APM 
should equal the number of APM s purchased. 

4. Select classes in which the screening tests will be given. The simplest arrangement 
is to select two or three teachers who between them teach the whole grade. For example, 
if everyone in the grade takes science, the test could be administered in science classes. 
Only two or three teacheis would be involved in the testing so that test administration 
would be fairly uniform. However, it would mean that the test was administered to students 
at various times of the day. Performance might not be too good by the last period of 
the day when students are tired or restless. 

Another approach would be to have the test given in, say, second period classes. 
This however, would mean involving many more teachers. 

Note that the use of the APM will give students a second chance so that small 
differences in performance due, for example, to being tired at the end of the day are 
not critical. 

5. When the tests arrive hold a meeting with the teachers involved in the 
administration who will be administering the SPM. Explain the testing program and the 
SPM and agree on a standardized testing procedure. This procedure could be simply the 
instructions given on page 9 on the SPM manual (Raven 1960). However, to generate 
enthusiasm it might be decided to use more sample questions, such as those reproduced 
in Appendix C of this report. These could be drawn on the blackboard in each classroom, 
reproduced on large poster boards by the art department or made into transparencies 
for use with an overhead projector. Items from set I of the APM could easily be copied 
for transparencies. It is recommended that only simple items, such as the first six of 
set I, be used since the purpose is to generate interest and confidence, not to teach students 
how to solve the items. 
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Whatever procedure is chosen, it should be uniform for all classrooms. Accurate 
timing of 30 minutes and care that no test booklet is lost should be stressed. 

Select testing days and arrange who will get the tests and answer sheets from 
whom, etc. 

Warn teachers that many stu-^ents, particularly in the lower ability classes, will 
finish before the time limit. They may wish to make some provision for this. 

Instruct teachers to tuni in after testing: 

(a) the answer sheets 

(b) a list of all students enrolled in the class as of the testing date. Students 
who missed taking the test, due to absence or some other eventuality, should be identified 
on this list 

C. When tests and answer sheets are turned in, check that no test booklets are 
missing. Have answer sheets screened and graded. 

Screening of answer sheets. A stencil made to fit over the answer sheet greatly 
speeds screening. In diecking answers start with E12, the most difficult item, and work 
backwards throu^ the test. A paper can be set aside when 15 items have been marked 
wrong since a score of 45 out of 60 is almost certain to be below the top six percent 
of scores. 

Grading of answer sheets . Score completely each paper in which less than 15 
items are missed. Record the total score (number right) on the class lists provided by 
the teachers. 

7. Amnge for the testing of those who missed the initial classroom administration 
of the SPM. Screen and grade these "pick-ups" and record on the class lists. 

8. From the class lists derive a rank ordei listing of the top students on the SPM. 
Test tlie top six percent with the APM. 

APM testing. The APM should be given to all selected students at the same 
time and supervised by a counselor. (For example, students can be summoned to the 
library for the APM.) Allow about 70 minutes total time. 

(a) Pass out set I and answer sheets. Have students answer set I. 

(b) When all students have finished, read a discussion of set I items. (A sample 
discussion is reproduced in Appendix C. Since it is necessary to point to items, 
transparencies for use on an overhead projector would be useful.) 
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(c) Pass out set IL Allow students to take as long as necessary to attempt 

all items. 

9. Grade set 11 of each APM paper and record the results on the rank order list 
of SPM scores. 

10. Determine the median score on the APM. Schedule for a WISC any student 
who was in the top half of the APM sample cr who was in the top two percent on 
the SPM. 

If time and money permit, schedule also those students nominated as possibly 
gifted by three or more teachers. 

1 1 . When all WISC results are available, select as the "special criteria mentally gifted" 
grc ip the students with the highest WISC performance scores. 
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APPENDIX C 



SAMPLE MATERIALS 



This appendix contains the following three samples: 



I * Questionnaire asking student opinions on 
tests. 

2. The explanations, given orally, after 
students had attempted Set I of the APM. 

3. Examples for mtroducing the SPM (XI, 
X2, etc.). 
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NAME 



TEST 



DATE 



PERIOD 



1. How much did you like this test? 



/ 

not at all 



/ 

OK 



/ 

very much 



2. Were the instructions clear to you? Did you always know what to do? 



/ 

instructions were 
hard. I didn't 
know what was 
wanted. 



/ 

OK 



3. How well did you understand this test? 



/ 

instructions were 
clear. I always 
knew what to do. 



/ 



/ 



/ 

OK 



/ 



/ 



not at all OK very well 

4. If the test were graded, what grade do you think you would get? 



/ 



/ 



F D 
5. Did you have enough time? 



/ 

C 



/ 
B 



/ 

A 



/ 

needed a 
lot more 
time 



/ 

needed a 
little more 
time 



/ 

had just 
enough 



6. Do you generally like taking tests? 



/ 

had time to 
finish easily 



/ 

too much 
time was 
given 



/ 



I hate tests 



/ 

no, not 
really 



/ 

OK 



yes 



I 

I love tests 
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THE EXPLANATIONS. GIVEN ORALLY, AFTER STUDENTS 
HAD ATTEMPTED SET I 



Now you've all finished trying set I; we'll take a look at some of the questions. 
I m going to read the explanations to you: this is to make sure everyone gets told the 
same things. 

Let's look at No. 5. If we look at the background first and read across the rows 
(pointing) we see that the background doesn't change. In the top row it's all white, the 
second row it's half shaded, in the third row it's all horizontally shaded. So we know 
the missing piece must have a horizontally shaded background. Now look at the foreground 
That changes with columns. There is nothing added on top of the background in the 
first column (pointing), in the second column there is a half vertically-shaded diamond, 
and m the last column there is a complete, vertically-shaded diamond. So the missing ' 
piece must have a complete vertically-shaded diamond as well as a horizontally shaded 
background. The correct answer, then is 2. 

Let's look at No. 7 now. In this one we can again separate foreground and background. 
The backgrounds stay the same across the rows so we will look for a solution which 
has a kind of graph-paper background. The foreground, in No. 7, is either a square, an 
X or a dot and there's one of each in each row, and in each column (check by pointing). 
So what is missing in the last row: a square, an X, or a dot? An X, right. So the answer 
was No. 6. 

Now No. 10. Do you see that by pushing together the first two figures in the top 
tow (pointing) we would get the third figure? It would also work for the second row, 
wouldn't it? So the answer is 8 because that is what we would get if we pushed the 
fust 2 figures of the last row together. We should check this answer to see that it works 
for the columns too. It does. 

In No. 11, we can apin think of the foreground and background separately. The 
foreground adds up along the rows, two dots and two dots make four dots (pointing) 
and the background adds up down the columns. The correct answer is 7. Always check 
both the rows and the columns. No. 12 was the most difficult. It can be thought of 
as a subtraction. Look at the middle column: if we take the second figure, the dot, away 
from the first figure, we'd be left with the third figure. This works for the first column 
too and also for the rows. So the correct answer is 6. The problem could also have been 
looked at as addition - add the second and third to get the first. There are often several 
ways to look at a problem, but always check that your answer works for rows and columns. 

The second booklet, set II, has the same kind of questions. There are 36 of them. 
You can take as long as you wish. Please mark down the time you start and finish. 
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APPENDIX D 
AN ATTEMPTED CROSS-VALIDATION 



Strong relationships can occur by chance, particularly when the number of 
observations is small. It is therefore essential to apply the measures to a different sample 
to see if the strong relationship is maintained, i.e., to cross-validate. 

Although funds remained for only eight more WISCs, it was decided to apply the 
recommended procedure to the eighth grade of a different school. 

Sample used in cross-validatio n. Two schools, school Y and school Z, close to the 
original school were available. School Y is on the borders of a white community and 
was one of the last schools in the district to become a predominantly minority school. 
The other school, Z, is located on the borders of an inner-city depressed area, and ail 
the elementary schools which feed into it receive title I funds. School Z was chosen for 
the cross-validation study. It was comparable to the original school in ethnic composition 
of the student body but smaller and located in a more depressed area. 

Procedures for cross-validation. The administration of the SPM was described to the 
two eighth grade science teachers in a brief after-school meeting. They administered the 
tests to each of their eighth grade classes. 

Papers from the 209 students were screened and about 30 students who scored over 
45 were located. Sixteen scored above 48 and were given the APM under the same 
conditions as in the original study. 

Papers had been screened down to a score of 45 in case the scores in this school 
were generally lower. It was encouraging that the high scores compared well with the 
scores from the original school. 

Out of the top ten students (all of whom scored 21 or higher on the APM and 
51 or higher on the SPM) eight were scheduled for testing with the WISC, but one earned 
the distinction of being the only student in the entire study who failed to appear to 
take the WISC. (All WISCs were given on a Saturday at the student's school.) 

Results . Table 24 shows the scores on the SPM, APM and, where available, the WISC. 
Whilst the matrices scores were as high as in the original study, the WISC scores were 
lower. It had been hoped that half of those tested would score at or above 114 on the 
performance scale and so be comparable to the WISC performance criterion group identified 
at the original school. Four of the seven tested scored 110 or better and if the standard 
error of estimate of the WISC is about five points (Anastasi 1961) there is a twenty 
percent chance that a person with a true score of 114 receives a score of 110 or less. 
However, the fact remains that aU the performance scale scores were below 114. How 
is this fact to be interpreted? 
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TABLE 24 

SPM, APM AND Wise SCORES FOR THE 
CROSS-VALIDATION SAMPLE 

Wise 

Student SPM APM Performance Verbal Full-Scale 



ZA 


55 


28 


1 lA 
1 lU 


1 nA 


lu / 


ZB 


54 


Z\J 


KIT 
IN 1 






zc 


52 


25 


lu/ 


yO 




ZD 


52 


23 


111 


104 


108 


ZE 


52 


21 


108 


104 


107 


ZF 


51 


23 


110 


92 


101 


ZG 


51 


24 


NT 






ZH 


50 


23 


NT 






ZI 


50 


23 


110 


97 


104 


ZJ 


50 


21 


99 


97 


98 


ZK 


50 


17 


NT 






ZL 


49 


19 


NT 






ZM 


49 


18 


NT 






ZN 


49 


15 


NT 






ZO 


48 


19 


NT 






ZP 


48 


18 


NT 







NT = Not tested 
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Possibly the screening and selection procedure is poor and students who would have 
scored over 1 14 were not selected for testing. Further testing at the same school would 
be needed to verify this and the implication would be that the results at the first school 
were due to very odd coincidences. 

On the other hand several other explanations can be offered for the lower WISC 
performance scores despite the equally high matrices scores. 

Firstly, there was a change in psychologist and, therefore, a possible examiner effect. 
The school psychologist who gave the original tests, a white male, is an experienced 
coureelor of teenagers, reputed to communicate very effectively. Unfortunately he was 
not available for the c.oss-validation study. The school psychologist who gave the 
cross-validation tests, a white female is not yet so experienced. As an indication that the 
students were not entirely at ease, she reported that only one student accepted any of 
the candy she made available. 

Another factor might have been the lower SES level of sdiool Z. It might be 
hypothesized that the WISC, which involves the child's dealing with time pressure and 
a strange adult, is more affected by SES level than the matrices. 
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